Joint Face Alignment with Non-parametric Shape Models

We present a joint face alignment technique that takes a set of images as input and produces a set of shapeand appearanceconsistent face alignments as output. Our method is an extension of the recent localization method of Belhumeur et al. [1], which combines the output of local detectors with a non-parametric set of face shape models. We are inspired by the recent joint alignment method of Zhao et al. [20], which employs a modified Active Appearance Model (AAM) approach to align a batch of images. We introduce an approach for simultaneously optimizing both a local appearance constraint, which couples the local estimates between multiple images, and a global shape constraint, which couples landmarks and images across the image set. In video sequences, our method greatly improves the temporal stability of landmark estimates without compromising accuracy relative to ground truth.